In recent years, an image processing technique using an imaging apparatus is applied in various fields, and there are many applications to such as a gene expression analyzer in medical fields. For example, there is a gene expression analyzer using a real time PCR method, a DNA micro array (also referred to as DNA chip), or a semiconductor nanocrystal.
A description will be given hereinafter of a gene expression analyzer using a fluorescence microscope as a prior art imaging apparatus.
Imaging targets by the prior art gene expression analyzer are beads having various spectral characteristics, each having a diameter of about 10 μm. A specific mRNA is combined with a bead having each spectral characteristic. The gene expression analyzer images beads and analyzes the spectral characteristics of each beads, and thereby identifies an mRNA that corresponds to the kind of the existing beads.
FIG. 13 is a block diagram illustrating a prior art gene expression analyzer using a fluorescence microscope.
In the prior art gene expression analyzer 600 shown in FIG. 13, there are provided a well plate 601 comprising a plurality of wells 602 for receiving a plurality of beads as observation targets, a well plate driving unit 603 for moving the well plate 601 in X and Y directions on a two-dimensional plane, a position reference imaging unit 630 for imaging silhouettes of the plurality of beads, a luminance reference imaging unit 640 for imaging luminance images of plural beads through plural optical filters each having a passing wavelength and different from each other, a CCD camera controller 611 for controlling a CCD camera 610, and a CPU 620 which analyzes the images imaged by respective imaging units 630 and 640 as well as controls the whole apparatus 600.
More specifically, the position reference imaging unit 630 includes an LED 606 as reference light, an objective lens 605, a z-axis driving unit 612 for moving the objective lens 605 in z-axis direction, a dichroic mirror 607 for reflecting a light of wavelength less than a predetermined value while passing a light of wavelength equal to or larger than the predetermined value, an imaging lens 609, and a CCD camera 610. A LED light from the LED 606 is applied to the plural beads as observation targets in the well 602, and the obtained silhouette lights of the beads are enlarged by the objective lens 605 to pass through the dichroic mirror 607 and the bandpass filter 608, and are collected by the imaging lens 609. Then, the z-axis driving unit 612 drives the objective lens 605 to align the focus position of the objective lens 605, and the CCD camera 610 images the silhouette lights to output a silhouette image as a two-dimensional image.
The luminance reference imaging unit 640 includes an excitation light source 613, an objective lens 605, a z-axis driving unit 612, a dichroic mirror 607, a filter wheel 614 which holds plural bandpass filters 608 each passing only a predetermined wavelength band, a filter wheel driving unit 615 which rotatably drives the filter wheel 614, an imaging lens 609, and a CCD camera 610. An excitation light from the excitation light source 613 is reflected by the dichroic mirror 607 and is applied to the plural beads as observation targets in the well 602 passing through the objective lens 605. The light generated in accordance with the spectral characteristic of the respective beads in response to the applied light are enlarged by the objective lens 605, and passes through the dichroic mirror 607 and the bandpass filter 608 to be collected by the imaging lens 609. Meanwhile, the z-axis driving unit 612 drives the objective lens 605 so as to align the objective lens 605 in its focus position, and then the CCD camera 610 images the light which is emitted from the respective beads and is collected by the imaging lens 609, to obtain a luminance image as a two-dimensional image.
Further, the CPU 620 includes a controller 621 which controls the whole apparatus 600, an analysis unit 622 which analyzes the two-dimensional image imaged by the CCD camera 610, and a mask image creation unit 623 which creates mask image that shows the existing area of the beads as imaging targets on the basis of the position reference image.
An operation of the prior art gene expression analyzer will be described. FIG. 14 is a flowchart illustrating a series of operations for obtaining the spectral characteristics of beads as observation targets in the prior art gene expression analyzer.
Initially, in step S101, position reference images for obtaining existing positions of plural beads as imaging targets are captured into the CPU 620 in the apparatus 600. To be specific, the controller 621 in the CPU 620 controls the well plate driving unit 603 so as to move the well plate 601 receiving the observation targets to be positioned right above the objective lens 605. Then, the controller 621 makes the LED 606 light to apply the LED light to the well 602. The LED light becomes the silhouette light for the beads as observation targets in the well 602, and the silhouette light is enlarged by the objective lens 605 and passes through the dichroic mirror 607 and the bandpass filter 608 to be collected by the imaging lens 609, and then reaches the CCD camera 610. The controller 621 instructs the z-axis driving unit 612 to align the objective lens 605 in its focus position so as to image the silhouette lights, and then instructs the CCD camera controller 611 to make the CCD camera 610 image the silhouette images of the plural beads as observation targets. Then, the analysis unit 622 in the CPU 620 binarizes the imaged silhouette images, and the binarized images are stored in the CPU 620 as position reference image for obtaining existing positions of the respective targets.
In step S102, a plurality of images which have passed through the respective optical filters are captured into the CPU 620 as images for obtaining luminance values of the respective imaging targets. To be specific, the controller 621 initially makes the LED 606 unlighted and makes the excitation light source 613 apply an excitation light. The excitation light is a light of short wavelength such as a blue laser beam. When the excitation light is incident on the dichroic mirror 607, due to the characteristic of the dichroic mirror 607 that it reflects a light of wavelength less than a predetermined value, the dichroic mirror 607 reflects the excitation light in the direction toward the objective lens 605. The objective lens 605 focuses the light from the dichromic mirror 607 on the observation targets in the well 602. The plural beads as observation targets existing in the well 602 present light emission patterns which respectively correspond to the spectral characteristics of the respective beads in response to the light applied from the objective lens 605, and the lights emitted from the respective beads pass through the objective lens 605, the dichroic mirror 607, and the bandpass filter 608, and further are collected by the imaging lens 609, and then reach the CCD camera 610 similarly as described above for the silhouette lights. At this time, since the bandpass filter 608 only passes a specific wavelength band, only the light of the specific wavelength band among the light emitted from the observation targets reaches the CCD camera 610. The controller 621 instructs the CCD camera controller 611 to make the CCD camera 610 image luminance images of only the specific wavelength bands having passed through the bandpass filter 608 among the light emitted from the observation targets. Then, the analysis unit 622 in the CPU 620 binarizes the imaged luminance images to be stored in the CPU 620 as luminance reference images.
In step S103, it is confirmed whether a predetermined number of luminance reference images obtained as above are captured or not, and when it does not yet reach the predetermined number, the controller 621 controls the filter wheel driving unit 615 to rotate the filter wheel 614 and to set the bandpass filter 608 passing a different wavelength band in the light path. Then, after performing the same processing as described above, the CCD camera 610 images a luminance image of a specific wavelength band which has passed through the newly set bandpass filter 608 among the light emitted from the observation targets. The analysis unit 622 then binarizes the imaged luminance image to be stored in the CPU 620 as a new luminance reference image. This processing is repeated a predetermined number of times until for example eight pieces of luminance reference images are obtained, and luminance reference images of various wavelengths are obtained.
After obtaining the position reference image and the luminance reference images as above, the processing transits to an analysis step of identifying the kind of the plural beads as observation targets using those reference image.
Here, the beads appearing on the respective luminance reference images and the beads appearing on the position reference image should be located at the same positions. Accordingly, in the analysis step, the respective luminance values of the respective beads are obtained for each optical filter from the respective luminance reference images, to identify the kind of the beads is identified on the basis of the luminance values.
Initially, in step S104, the mask image creation unit 623 in the CPU 620 creates a mask image indicating the bead presence areas using the captured position reference image.
FIG. 15 is a diagram illustrating a mask image and luminance reference images. In FIG. 15, reference numeral 801 denotes a mask image showing bead presence areas, which is obtained by performing a masking processing that masks higher luminance portions at the center portions of the beads in the position reference image.
In FIG. 15, reference numeral 701a denotes a first luminance reference image obtained after passing through the bandpass filter 608 that passes a light of 505 nm wavelength, the reference numeral 701b denotes a second luminance reference image obtained after passing through the bandpass filter that passes a light of 525 nm wavelength, and the reference numeral 701c denotes a third luminance reference image obtained after passing through the bandpass filter that passes a light of 545 nm wavelength.
Here, 8 pieces of bandpass filters 608 that respectively passes the lights having wavelengths different from each other by 20 nm are used to obtain 8 pieces of luminance reference images in total.
FIG. 15 shows only first three pieces among 8 pieces of luminance reference images.
The areas B1m, B2m, and B3m on the mask image 801 are bead presence areas where the beads B1, B2, and B3 are present, respectively, and the areas B1a to B1c, the areas (B2a to B2c) (do not appear in FIG. 15), and the areas B3a to B3c on the first to third luminance reference images 701a to 701c are areas on the luminance reference images which correspond to the areas B1m, B2m, and B3m on the mask image 801, respectively.
In steps S105 to S106, assuming that the positions of the bead areas B1a to B1c, B2a to B2c, and B3a to B3c which are present on the respective luminance reference images are the same as the positions of the bead area B1m, B2m, and B3m which are present on the mask image 801, respectively, the analysis unit 622 in the CPU 620 obtains the respective luminance average values of the areas B1a to B1c, B2a to B2c, and B3a to B3c on the respective luminance reference images.
Assuming, for example, that a luminance average value of the area B1a on the first luminance reference image 701a is A, a luminance average value of the area B1b on the second luminance reference image 701b is B, and a luminance average value of the area B1c on the third luminance reference image 701c is C, in step S107, the luminance average values obtained as above are plotted to result in FIG. 16.
FIG. 16 is a diagram illustrating the plotted luminance average values of the respective areas on the luminance reference images, which areas correspond to the three bead areas on the mask image.
In FIG. 16, the abscissas indicates a wavelength transmitting through the bandpass filter while the ordinates indicates the luminance average value. The reference numeral 901 indicates a spectral curve indicating the characteristic of beads B1, obtained by plotting the 8 luminance average values of the areas on the first to eighth luminance reference images corresponding to the bead presence area B1m and connecting the plot points. Reference numeral 902 indicates a spectral curve indicating the characteristic of beads B2, obtained by plotting the respective luminance average values of the areas on the luminance reference images corresponding to the bead presence area B2m and connecting the plot points. Reference numeral 903 indicates a spectral curve indicating the characteristic of beads B3, obtained by plotting the respective luminance average values of the areas on the luminance reference images corresponding to the bead presence area B3m and connecting the plot points.
In step S108, the analysis unit 622 analyzes the spectral characteristics of the respective beads B1 to B3 on the basis of the respective spectral curves 901 to 903 thus obtained and identifies the kinds of the beads, respectively.
In the conventional method, as shown in FIG. 17(a), assuming that, when the mask image is overlaid on the first luminance reference image, the bead areas B1m to B3m on the mask image and the bead areas B1a to B3a on the first luminance reference image are located at the same positions, respectively, luminance average values of the respective areas on the luminance reference images corresponding to the bead presence areas B1m to B3m on the mask image are obtained and the kinds of the beads are respectively identified on the basis of the average values.
In this conventional method, however, there is no means for detecting vibrations, and even when vibrations occur while the luminance reference image is being imaged and thereby the position of the beads on the luminance reference image is changed, the processing is performed similarly as described above. Therefore, when the mask image is overlay-displayed on the luminance reference image and a deviation has occurred between the beads presence areas B1m to B3m on the mask image and the bead areas on the luminance reference images as shown in FIG. 17(b), the analysis unit 622 cannot obtain luminance average values of the respective beads on the respective luminance reference images correctly, and thereby the kinds of the beads cannot be properly identified.
FIGS. 17(a) and 17(b) are diagrams illustrating relationships between the beads areas present in the mask image and the bead areas present in the first luminance reference image, wherein FIG. 17(a) shows a case where the mask image is overlaid on the luminance reference image in its display when no vibrations occur and FIG. 17(b) shows a case where the mask image is overlaid on the luminance reference image in its display when vibrations occur.
To solve this problem, it may be conceived to mount a vibration detection sensor close to the imaging target and to judge whether the positions of the beads has deviated or not before or during imaging the respective luminance reference images employing the technique disclosed in the Japanese Published Patent Application No. Hei.5-130545.
However, when the vibration detection method as disclosed in the above patent reference is employed, it is necessary to provide a vibration sensor in the apparatus 600, which results in an increase in cost as well as necessitating a large amount of labor for interrelating the vibrations and the deviations in the beads positions.
This comes from that as long as a detailed analysis is not made as to the amplitude and the direction of the vibrations, a mounting position of a sensor, and relationships between the vibrations and the sensor positions, it is difficult to properly relate the allowable range for the sensor output and the allowable range for the actual beads position deviations.